feat(core): add user transcription timeout#6182
Conversation
chenghao-mou
commented
Jun 22, 2026
chenghao-mou
commented
- add a new event when vad detects any speech, but stt fails to produce any transcript during that timeout
add a new event when vad detects any speech, but stt fails to produce any transcript during that timeout
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
🚩 New event not forwarded in SessionHost remote transport
The SessionHost in remote_session.py registers handlers for specific events (lines 371-379) and forwards them over the transport. The new user_transcription_timeout event is not registered or forwarded. This means remote sessions won't receive this event. This is likely acceptable as a first iteration (not all events need remote transport support immediately), but it's an inconsistency with the event being in EventTypes and AgentEvent.
(Refers to lines 366-379)
Was this helpful? React with 👍 or 👎 to provide feedback.
| def _on_transcription_timeout(self) -> None: | ||
| self._transcription_timeout_handle = None | ||
| if self._user_turn_start is None or self._turn_transcript_received: | ||
| return | ||
|
|
||
| if self._agent_speaking: | ||
| return | ||
|
|
||
| self._hooks.on_transcription_timeout( | ||
| speech_duration=self._turn_speech_duration, turn_start=self._user_turn_start | ||
| ) |
There was a problem hiding this comment.
🚩 Transcription timeout suppressed during agent speech with no re-arm
When _on_transcription_timeout fires while the agent is speaking (audio_recognition.py:1809), the callback returns silently without emitting the event and without scheduling a retry. This means if VAD detects user speech that produces no transcript, and the timeout happens to fire while the agent is still speaking (e.g. the user spoke over the agent near the end of its turn), the timeout event is permanently lost for that speech burst. The rationale is likely that speech detected during agent output is often echo/noise, but with AEC enabled it could be genuine user speech. Whether this is acceptable depends on the use case.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Since there is no user content, we should still emit the signal so the agent can check.
| self._user_speaking_event.clear() | ||
| self._last_speaking_time = time.time() - ev.silence_duration - ev.inference_duration | ||
|
|
||
| self._arm_transcription_timeout(ev.speech_duration) |
There was a problem hiding this comment.
maybe skip this when stt is not set?
|
should we have a default handler with the above implementation? any downsides of having this work automatically? |
I am wondering how many false alarms it may have, it can by risky if there are some background noise. |
3 participants
